Pharmacological Modulation of Temporal Discounting: A Systematic Review

Temporal discounting is a phenomenon where a reward loses its value as a function of time (e.g., a reward is more valuable immediately than when it delays in time). This is a type of intertemporal decision-making that has an association with impulsivity and self-control. Many pathologies exhibit higher discounting rates, meaning they discount more the values of rewards, such as addictive behaviors, bipolar disorder, attention-deficit/hyperactivity disorders, social anxiety disorders, and major depressive disorder, among others; thus, many studies look for the mechanism and neuromodulators of these decisions. This systematic review aims to investigate the association between pharmacological administration and changes in temporal discounting. A search was conducted in PubMed, Scopus, Web of Science, Science Direct and Cochrane. We used the PICO strategy: healthy humans (P-Participants) that received a pharmacological administration (I-Intervention) and the absence of a pharmacological administration or placebo (C-Comparison) to analyze the relationship between the pharmacological administration and the temporal discounting (O-outcome). Nineteen studies fulfilled the inclusion criteria. The most important findings were the involvement of dopamine modulation in a U-shape for choosing the delayed outcome (metoclopradime, haloperidol, and amisulpride). Furthermore, administration of tolcapone and high doses of d-amphetamine produced a preference for the delayed option. There was a time-dependent hydrocortisone effect in the preference for the immediate reward. Thus, it can be concluded that dopamine is a crucial modulator for temporal discounting, especially the D2 receptor, and cortisol also has an important time-dependent role in this type of decision. One of the limitations of this systematic review is the heterogeneity of the drugs used to assess the effect of temporal discounting.


Introduction
Intertemporal decision-making is choosing between different outcomes at different times [1]. A reward or outcome will lose value while time passes; this phenomenon is known as 'temporal discounting' or 'delayed discounting' [2]. As the value of the delayed in-time outcome is discounted, there is a higher bias for the sooner choice [3]. This decision is prevalent in daily life, specifically in the areas of health, education, investment, and even clinical conditions [4].
in-time outcome is discounted, there is a higher bias for the sooner choice [3]. This decision is prevalent in daily life, specifically in the areas of health, education, investment, and even clinical conditions [4].
Temporal discounting is measured with tasks in which participants have to choose between a smaller amount of money delivered immediately or after a short amount of time and a large amount of cash given in the future. e.g., 'Would you prefer 40 dollars today or 100 dollars in three months?'. A person has a higher discount rate when his or her preference goes towards the smaller and more immediate reward. On the other hand, the person has a lower discount rate when he has a stronger preference for the delayed and bigger reward [5].
Thus, the preference for the small, immediate reward over the higher, delayed one is considered an impulsive decision. The preference for the higher, delayed reward over the smaller, immediate reward is a self-controlled decision [6]. Non-human animals prefer impulsive decisions, such as monkeys, rats, mice, and pigeons [1]. The discounting is best described by a hyperbolic curve that shows how the reward loses value with the passage of time ( Figure 1). Furthermore, the tasks that measure temporal discounting have a variable known as the k-value. It is a free parameter that measures the sensitivity to the delay. When the k value is low, it means the individual discount is lower with the delay in time, and when the k-value is high, the individual is susceptible to the delay [7]. This type of decision has higher discount rates in many pathologies such as addictive behavior [8], heroin addiction [9], cocaine addiction [10], alcohol addiction [11], opioiddependents [12], smokers [13], bipolar disorder [14], attention-deficit/hyperactivity disorder [15], antisocial personality disorder, social anxiety disorder [16], borderline personality disorder [17], major depressive disorder [18], schizophrenia and schizoaffective disorders [19], pathological gambling, and orbitofrontal cortex lesions [20].

Methods
This systematic review followed the PRISMA guidelines [21]-[see Supplementary Material].

Eligibility Criteria and Search Strategy
The PICO strategy was used to identify relevant studies in the question being investigated. PICO is an acronym for Population (P), intervention (I), Comparison (C), and Outcome (O). This systematic review used Healthy Humans (P), Pharmacological administration (I), and Intertemporal decisions (O). The exclusion criteria considered review articles, study cases, book chapters, research in animals, letters to the editor, editorials, protocols, pilot studies, and studies without the information required in the question.
The guide question for this review was the following: Is temporal discounting affected by pharmacological interventions? The search was completed in various databases, including PubMed, Scopus, Web of Science, Cochrane, Science Direct, and Google Scholar. The Mesh terms, keywords, and search strategies were organized according to each database and are included in the Supplementary materials.
As of the start of the research, we had programmed an alert for new articles in each database. There were no time or language restrictions in the search for articles. Two reviewers performed the search and selection of the studies, the extraction of the data, and the risk of bias analyses as a double-blind evaluation. The references were uploaded into the citation manager (EndNote ® , version X9, Thomson Reuters, Philadelphia, PA, USA, EUA).

Data Extraction
The EndNote ® removed the duplicate references first automatically, followed by the authors conducting it manually. The studies were screened by title and abstract. The remaining ones went through a full-text review, and the articles that did not fulfill the inclusion criteria were removed. From the selected studies, we extracted: authors, country, publication year, type of study design, participant characteristics (sample size), age, mean, intertemporal decision task, pharmacological kind of intervention/dose, and results (Table 1).  Delay and probability discounting task (DPD [9]).
There were no discernible trends of an effect of oxycodone on the DPD.

Data Analysis
We chose Rob 2.0-a revised Cochrane risk of bias tool for randomized trials-to evaluate the risk of bias. It is a reviewed tool used and adapted to assess different types of study interventions. The Rob 2.0 proposes a series of questions based on five domains of bias and an overall evaluation. The domains are deviation from intended intervention, missing outcome data, measurement of the outcome, selection of the reported result, and overall. Each domain presents some questions related to the trial design, conduction, and reporting of the results. The question is answered as Yes, Probably Yes, No, or Probably No. These responses stratify the domains as High Risk if there are significant problems, 'Some concerns' if there are minor problems, or 'Low Risk' if there are no problems.

Selection of the Studies
The database search identified 2065 references, of which 477 were duplicate references. The remaining 1588 references were analyzed by title and abstract. Twenty-three articles fulfilled the inclusion criteria and went to full-text analysis.
From the twenty-three articles assessed by full-text analysis, eight were excluded, seven of them because they did not have a pharmacological intervention and one because it used rats as subjects. Finally, nineteen studies fulfilled the inclusion criteria and went through the quality review. Each step of the review was performed independently by two reviewers, and a third reviewer reviewed the included articles. The steps are described in the PRISMA flowchart ( Figure 2). through the quality review. Each step of the review was performed independently by two reviewers, and a third reviewer reviewed the included articles. The steps are described in the PRISMA flowchart ( Figure 2).
Used to treat schizophrenia. Also, symptoms of agitation, irritability and delirium

Metoclopramide
Metoclopramide is a dopamine D2 antagonist. It inhibits dopamine D2 and serotonin 5-HT3 receptors in the chemoreceptor trigger zone located in the area postrema of the brain.
Used to treat gastroesophageal reflux disease, and prevention of nausea and vomiting Levadopa Levadopa is a dopamine precursor. While dopamine cannot cross the blood brain barrier, Levadopa is able to. After crossing the barrier Levadopa is converted to dopamine Used in the treatment of attention deficit hyperactivity disorder and narcolepsy.

Yohimbine
Yohimbine is a pre-synaptic alpha 2-adrenergic blocking agent. It increases norepinephrine release Yohimbine is found in supplements.

Propranolol
Propranolol is a non-selective beta-adrenergic receptor antagonist. Means that it does not have preference for Beta receptors. It inhibits sympathetic stimulation of the heart. It reduces resting heart rate, cardiac output, blood pressure.

Hydrocortisone
Hydrocortisone, or cortisol, is a glucocorticoid secreted by the adrenal cortex. It is essential for life and supports many important cardiovascular, metabolic, immunologic, and homeostatic functions Used to treat corticosteroid-responsive dermatoses, endocrine disorders, immune conditions, and allergic disorders.

Testosterone
Testosterone is a steroid sex hormone Used to treat hypogonadism or breast carcinoma in women,

Marinols (Dronabinold)
Marinol (Dronabinol) is a cannabinoid. It's a synthetic form of THC, which is an ingredient found in marijuana.
Treatment of anorexia and weight loss in people with acquired immunodeficiency syndrome. Also is used in treatment of nausea and vomiting from chemotherapy Diazepam Diazepam is a benzodiazepine that binds to receptors in the brain and spinal cord. This binding increases the inhibitory effects of GABA (gamma-aminobutyric acid). GABA is involved in sleep induction, memory, anxiety, epilepsy and neuronal excitability.
Used to treat panic disorders, severe anxiety, and seizures.

Oxycodone
Oxycodone is an opioid. It binds to the mu opioid receptor, and to the kappa and delta opioid receptor. Used in the treatment of pain.

Naltrexone
Naltrexone is a opiate antagonist. It may block the effects of endogenous opioids. Used in opioid overdose.

Risk of Bias
The risk of bias tool was an adapted version of Rob 2.0 that evaluated five domains of the studies: randomization process, deviation from the intended intentions, missing outcome data, measurements of the outcome, and selection of the reported results. Most of the studies presented a low risk of bias in all the domains. However, one of the studies showed some concerns in one of the domains (Figure 3).

Discussion
This systematic review aimed to understand the association between the administration of some drugs and temporal discounting (see Table 3). Thus, nineteen studies fulfilled the inclusion criteria of the systematic review, of which eighteen presented a low risk of bias. Five studies that administered d-amphetamine (20 mg) [28], tolcapone (200 mg) [31], metoclopramide (10 mg) [26], haloperidol (2 mg) [38], and amisulpride (400 mg) [39] reported an increment at choosing the delayed reward. Four studies with drugs administered, such as hydrocortisone 10 mg [2.,32], androgel 150 mg [41], and l-dopa 150 mg [36], found an increase in the preference for the more immediate reward. The administration of haloperidol 1.5 mg [36], pramipexole 0.25 mg and 0.50 mg Thus, eighteen of the studies were evaluated as having low Risk of bias in all domains Acheson et al. [22], Acheson [38], and Zacny and de Wit [42]. Nevertheless, McDonald et al. [33] reported some concerns of bias in the domain of missing outcome data because the data for all the participants regarding the temporal discounting task were not available for analysis.
A systematic review is a set of scientific strategies applied to synthesize and analyze the best evidence on a specific topic. It summarizes and compiles the scientific evidence and decreases the risk of bias in the studies [44]. To achieve this, a quality assessment using a specific tool according to the types of studies included in the review was performed [45]. The systematic review gives a clear view of a topic, helps make decisions related to health and research, establishes new policies, and plans future research [44].

Dopaminergic System
Dopamine is a crucial neurotransmitter for animals that have been involved in a variety of functions and behaviors. This neuromodulator has been strongly associated with impulsivity and clinical disorders involving impulsive behaviors. Preclinical and human studies suggest a crucial role for dopaminergic function in temporal discounting. Parkinson's disease patients, a disease known for dopamine deficiency, have shown altered temporal discounting compared with healthy volunteers [46]. Some of the drugs that act in the dopaminergic system, such as Tolcapone and L-dopa from the included studies are employed in the treatment of Parkinson's disease.
There is a consistent finding within the studies. The dopamine receptor D2 modulation is involved in choosing the later reward over the sooner reward [26,38,39]. Three drugs act on the D2 receptor: amisulpride [39], haloperidol [38], and metoclopramide [26]. This D2 receptor has been associated with impulsive behavior.
In mice, the absence of the D2 receptor increased impulsive behavior, while restoration of the expression of the D2 receptor decreased impulsivity [47]. Another study knocked down these receptors in rats and found that these knockdown rats had a higher preference for the smaller and immediate reward than the control rats in a delay-discounting task [48]. Furthermore, lower ventral striatal D2 density is associated with impulsivity and greater temporal discounting, i.e., preference for the sooner option [49].
Following this reasoning, a study with methamphetamine-dependent subjects found they had lower striatal dopamine D2/D3 receptor availability than healthy controls, and higher impulsiveness was related to it [50]. Furthermore, another study found that methamphetamine-dependent subjects with lower striatal D2/D3 receptors have steeper temporal discounting [51].
A meta-analysis confirmed that Parkinson's disease patients have a steeper discount than healthy controls [52], emphasizing the crucial role of dopamine in this type of decision. Additionally, a study found that Parkinson's disease patients treated with dopaminergic drugs exhibited greater farsightedness in their choices than Parkinson's disease patients not on medication [53].
Wagner et al. [38] and Pine et al. [36] used the same drug, Haloperidol, where Pine et al. [36] found no effect of the drug. It should be noted that this difference could be due to the differences between the doses used in both studies: 2 mg [38] and 1.5 mg [36]. It is possible that lower doses of this dopamine antagonist produce a different result and that the effect of the drug would be dependent on dopaminergic baseline levels. It is hypothesized that dopamine has an inverted U-shape function between dopamine and impulsivity, where both extremes are associated with the worst decision-making performance [54]. It is consistent with results from the Castrellon et al. [55] meta-analysis, which found the same results in rats.
Hamidovic et al. [29] administered Pramipexole, a dopamine D3 receptor agonist, and found the drug had no effect. This result suggests D3 is not related to temporal discounting, but, as the author noted, it remains to be seen if multiple doses over a long period of time or higher doses present a different effect. Interestingly, Parkinson s disease patients treated with Pramipexole have shown a proportion of impulse-control disorder related to the D3 receptors [56]. This has been seen even in rats where the D3 receptor is related to preferring the larger and delayed reward [57].

D-Amphetamine
D-Amphetamine is a drug known to have a high potential for abuse but is also used in the treatment of adults with Attention-Deficit Hyperactivity Disorder ADHD [58]. It can produce a feeling of well-being and euphoria and has helpful behavioral effects on ADHD patients. It increases dopamine in the synapses by binding to the dopamine transporter DAT, reversing DAT function in the medial prefrontal cortex, and inhibiting dopamine uptake [59]. Bupropion is commonly used to decrease impulsivity [60]. Bupropion is a drug that improves impulse control and attention in some patient populations, including those with ADHD [61]. Bupropion is a norepinephrine/dopamine-reuptake inhibitor. The administration of Bupropion had no effect on temporal discounting [25], the same as the administration of d-Amphetamine [25] and a low dose (10 mg) of d-Amphetamine [28]. However, administration of a higher dose (20 mg) in the same sample produced a preference for the later reward [28]. The higher dose by de Wit et al. [28] and the dose used by Acheson et al. [25] were the same (20 mg). However, the results differ, making the relationship between the acute pharmacological relationship and the task unclear. This is highlighted even more with the decrease in impulsivity (e.g., Go/No-Go task, Stop Task) found as other impulsivity measures were applied [28]. This could be explained by the dopamine U-shape action, as has been observed in mice and rats after d-amphetamine administration, where the shifted their response toward the sooner choice [62,63].
Another possible hypothesis Maguire et al. [64] found was that the effects of amphetamine differ depending on the manner in which the delayed reward is presented. This suggests that the changes in the results of the studies and the diverse performance were in part due to the sensitivity to the reward delay.

Levodopa (L-Dopa)
L-dopa is a dopamine precursor that passes the blood-brain barrier and converts it into dopamine. Dopamine is not able to pass the blood-brain barrier. This drug is often used in the treatment of Parkinson s disease. The results of this review showed that Pine et al. [36] found that L-dopa had an effect on preference for a sooner reward, whereas Petzold et al. [35] did not find any effect.
A secondary result from Petzold et al. [35] analyzed that low-impulsive individuals exhibit a preference for the sooner reward, converging with Pine et al. [36] results, but more impulsive individuals showed the opposite effect. These could be explained by the dopamine baseline levels, because, as mentioned, before dopamine behaves in a U-shape. Another possible hypothesis to explain this different result could be the difference in sample size between the two studies; Petzold et al. [35] had a sample size of 87 participants, while Pine et al. [36] had 14 participants. Animal studies have found that L-dopa leads to impulsive-like behaviors [65] and increases impulsivity in Parkinson s disease even when improving some cognitive tasks [66].

Tolcapone
Tolcapone is a drug that inhibits the enzyme catechol-O-methyl transferase (COMT). This drug is used in the treatment of Parkinson s disease. By inhibiting COMT, the degradation of L-dopa is prevented, permitting higher concentrations to cross the bloodbrain barrier, and become dopamine [67]. After the administration of Tolcapone, Kayser et al. [31] found an increased preference for the later reward.
Congruently with this result, the genotype for the enzyme COTM predicts impulsive choice behavior. Specifically, subjects carrying enzymatically fewer active alleles encoding the COMT gene show a decrease in choosing the immediate reward [68,69]. Tolcapone increases dopamine tone prefernetially in the frontal cortex, and COMT is mainly in charge of degrading dopamine in the frontal cortex. Thus, low dopamine levels in the frontal cortex may predispose to higher impulsivity.

Hypothalamic-Pituitary-Adrenal Axis (HPA-Axis)
The HPA axis is a major neuroendocrine system that aims to maintain physiological homeostasis and modulates many important processes such as the stress response, metabolism, fertility, and immunity. The hypothalamus secretes the corticotropin-releasing hormone (CRH), which releases the Adrenocorticotropic hormone (ACTH) from the pituitary gland. This stimulates the production of corticosteroids from the adrenal gland. Cortisol is the main corticosteroid and a stress biomarker. Cornelisse et al. [27] and Riis-Vestergaard et al. [2] administered hydrocortisone and found that after 15 min there was a change in the preference for the sooner reward but not the later. These results suggest a link between HPA axis activation and the mentioned acute stress-induced studies [3,32], where similar results were found. Administration of exogenous cortisol and increasing levels of endogenous cortisol produce the same response in intertemporal decisions. It should be noted that this response is time-dependent, as measures taken 185 min later did not reveal the results.

Sympathetic-Adreno-Medullar System (SAM-System)
Herman et al. [30] and Lempert et al. [32] administered drugs that modulate the SAM system. The former used yohimbine to stimulate the system, and the latter used propranol to suppress it. Both found no effect. The Sympathetic-Adreno-Medullar System is part of the sympathetic nervous system. It gives a quick physiological response to face the challenge of outside stimuli. This reaction is mediated by catecholamines, especially epinephrine and norepinephrine, and leads to an increase in the heart rate and blood pressure. It is also one of the major systems involved in the stress response, along with the Hyphotalamic-Pituitary-Adrenal axis. Some studies administering protocols that produce acute stress have found that stressed individuals prefer the earlier option over the delayed option [3,70]. These pharmacological results could suggest that the SAM-system is not the main reason for the choice selected, but the HPA-axis should be further investigated.

Testosterone
Testosterone is a major sex hormone in men and women [71]. The two studies that looked for the effects of testosterone in temporal discounting found different results. While Wu et al. [41] found a preference for the smaller reward, Ortner et al. [34] found no effect. These differences could be dose-dependent; Wu et al. [41] administered 150 mg of testosterone, whereas Ortner et al. [34] used 50 mg. A meta-analysis reported a positive association between circulating testosterone and impulsivity [72]. This relationship could be possible due to the effects of testosterone in the dopaminergic system, as testosterone receptors are in dopaminergic neurons that project to the ventral striatum [73].

Opioids and Endocannabinoids
Opioidergic and cannabinoid systems involved receptors located throughout the brain and body. The opioid system is involved in affective processing, pain, pleasure, and reward [74]. The endocannabinoid system is involved in regulating physiological and cognitive processes, appetite, pain sensation, mood, and the pharmacological effects of cannabis [75]. Two studies administered drugs that act in the opioid system [39,42], and one in the endocannabinoid system [33]. All these studies were looking for the effects of these systems that have been implicated in drug addiction. None of the studies found any effect of the drugs used in participants.
These findings in the opioid system differ from preclinical studies in rats where the opioid receptor agonist, morphine, increased impulsive behavior in a temporal discounting task [76,77]. This difference could be explained by considering the differences between the tasks measured in the clinical and preclinical models. In the rats' studies, the rats performed the task when the animal was in a drugged state, whereas in humans, the measure could be after the drug's effects have dissipated. Furthermore, it is possible the temporal discounting is unaffected by drug effects, even when they affect other types of impulsivity. Some other acute administrations of drugs have failed to affect temporal discounting, for example, alcohol [78] and benzodiazepines [13].
McDonald et al. [33] also found no effect after the administration of THC. It was thought that THC would have a relationship with impulsivity modulated by the activity of dopaminergic neurons. THC and other cannabinoids increase the activity of dopaminergic neurons [79]. Moreover, earlier research found that marijuana alters the perception of time [80]. The null findings in the McDonald et al. [33] study may be due to the participants being required to decide on a reward that will be obtained after the effect of the drug has dissipated, as with opioid drugs.

Diazepam
Diazepam is a benzodiazepine that increases the inhibitory effects of gammaaminobutyric acid (GABA). Some of the GABA functions are sleep induction, memory, anxiety, and epilepsy. Diazepam, as well as other benzodiazepines, are used for the treatment of anxiety, as a muscle relaxant, as an anticonvulsant, and sometimes are abused [40]. Acheson et al. [22] and Reynolds et al. [13] found no effect of diazepam in temporal discounting.
These results are divergent from the literature that suggests Diazepam affects impulsive behavior in human and non-human models [13,81]. Deakin et al. [82] found in healthy volunteers that after administration of 29 mg of diazepam there were disinhibitory cognitive effects. There are reports that benzodiazepines produce disinhibition and increase aggression [83], and even benzodiazepines such as flunitrazepam, increase impulsivity and aggression [84]. Interestingly, it appears that Diazepam just increases some specific types of impulsive behavior but not others. Acheson et al. [22] found that Diazepam affects the performance of impulsive behavior in the stop task and go/no-go task but not the temporal discounting task.
All the studies in this systematic review went through a risk of bias analysis. Eleven studies were within the design interventions, and eight studies were placebo-controlled interventions. That makes Rob 2.0 an effective tool for the evaluation of the risk of bias and methodological assessment. All the studies presented a low risk of bias, except McDonald et al. [33], which had some concerns about missing outcome data.

Limitations
The findings of this study have to be seen in light of some limitations. While this systematic review followed a pre-registry strategy, recommended guidelines, and used a theory-based interpretation of the results, there are some limitations and considerations in this study that could be addressed in future research. First, the drugs administered to assess the effect on temporal discounting were extensively different, both in type of drug and dosage. This led us to decide against conducting a quantitative assessment of the evidence (meta-analysis), which in turn poses a limitation to the confidence of our conclusions.
Furthermore, the small sample size of the studies included and the use of a k-value for the analysis should be taken into consideration. A potential methodological issue may be present when using the k-value on temporal discounting as it may affect the results, as seen in McDonald et al. [33], where the sample strongly decreases due to not fitting the k values.

Conclusions
This systematic review allowed us to observe the effect of different types of drugs on Temporal Discounting. It showed that some drugs could influence the intertemporal decision outcomes in humans, either by lowering or by augmenting them. There is also evidence for possible clinical pathways to prevent or treat impulsiveness. This review points out that drugs involved with the dopaminergic systems exhibited changes in behavior. Especially, receptors D2 and D3 are related to the choice of the later reward. Furthermore, drugs that involve hormonal systems such as testosterone and cortisol impact temporal discounting, leading to more impulsive choices. It's also important to keep in mind that cortisol's impact on temporal discounting is time-dependent. Further research will continue to help understand the underlying physiological and neurological mechanism of temporal discounting and how this can be affected by the intake of drugs that lead the subject to be more impulsive or self-controlled.